Skip to content

fix(schema): refuse private field placeholders in file_pattern (#1639) - #1643

Merged
markramm merged 4 commits into
pyrite-wiki:devfrom
Mohit-Gaur75:fix/1639-file-pattern-refuse-private-fields
Nov 17, 2026
Merged

markramm merged 4 commits into
pyrite-wiki:devfrom
Mohit-Gaur75:fix/1639-file-pattern-refuse-private-fields

Conversation

@Mohit-Gaur75

@Mohit-Gaur75 Mohit-Gaur75 commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #1639.

In FieldSchema.resolve_filename (pyrite/schema/field_schema.py):

  • Collect dataclass_fields and split into data_fields and private_fields (f.name.startswith("_")).
  • Refuse only when field_name in private_fields, raising ValidationError(f"file_pattern {self.file_pattern!r} names {{{field_name}}}, which is a private field of entry {entry.id!r} (type {entry.entry_type!r}); refusing to resolve it").
  • Metadata keys starting with _ (e.g. _owner, _draft on kb.yaml-declared types) continue to resolve from entry.metadata, and absent _-prefixed names fall back to default filename (None).
  • Replaced the single fixture test with the structural test: parametrized over Entry and all subclasses reachable via Entry.__subclasses__() crossed with each class's _-prefixed dataclass fields (143 pairs), asserting refusal. Added four control tests (public fields resolve, _draft metadata key on generic entry resolves, absent _ name returns None, and {date}--{slug}.md unchanged) and a test through KBService.create_entry.
  • Added changelog fragment changelog.d/1639-file-pattern-private-fields.fixed.md.

Changelog: added

@markramm markramm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 8a0933b (conductor, 2026-11-17)

Automated: an Opus reviewer read the diff cold (the resolver, its caller, the entry dataclasses and the existing tests), then read it against the groom on #1639; the conductor checked its findings against the code. Read-only: nothing was run. CI on this head had not finished when this was written.

Thank you for picking this up within the hour. The refusal is in the right place: the one resolver, after the plain-name check and before any value is read, so a dry run and a real write answer the same. It reuses the existing ValidationError, and format-spec spellings such as {_x:>3} are covered.

The change refuses more than the issue asks, and that is the blocking part.

Blocking

  1. pyrite/schema/field_schema.py, resolve_filename: refuse only a private field of the entry, not every name that starts with _. The new branch if field_name.startswith("_"): raise runs before the metadata lookup. That is option B in the groom, which it recommends against. Two things that work on dev stop working:

    • A kb.yaml type whose entries carry the author's own key _owner: alice and whose pattern is {_owner}-{slug}.md. On a kb.yaml-declared type, unknown frontmatter keys live in entry.metadata, and dev resolves the name from there (elif field_name in meta). With this change every new entry of that type is refused, and the message says {_owner} "is not a data field", although it is the author's own data.
    • A _-prefixed name the entry does not have at all ({_custom_private}). On dev that returns None, the documented fallback to the default file name for an absent field. Here it raises.

    What the groom asks for (option A): refuse when the name starts with _ and is in {f.name for f in dataclasses.fields(entry)}. Leave _ metadata keys resolving and absent _ names falling back, as every other name does. With that condition, the if not f.name.startswith("_") filter you added to data_fields is no longer needed: keep one of the two, not both.

  2. tests/test_file_pattern.py: the structural test from the groom's acceptance. The current test loops over four hand-written names on ADREntry; the first failure hides the rest, and _importance_given and _custom_private are not fields on dev, so they only exercise the any-_ rule. Wanted:

    • parametrise over Entry and every class reachable by recursing Entry.__subclasses__() (import the model modules first so GenericEntry and the task entry are loaded), crossed with [f.name for f in dataclasses.fields(cls) if f.name.startswith("_")]; for each pair assert TypeSchema(name="t", file_pattern="{" + name + "}.md").resolve_filename(instance) raises ValidationError;
    • a guard that the collected list is not empty;
    • controls: a public field still resolves; a metadata key _draft on a kb.yaml-declared type still resolves; an absent _ name still returns None; {date}--{slug}.md is unchanged.

    A new private field on any entry class is then refused without anyone editing the test. This also pins the groom's extra finding, _entry_type on GenericEntry, which no test builds today.

  3. A changelog fragment. "Changelog: none" does not fit: a write now refuses a pattern it used to accept. Add changelog.d/1639-<slug>.fixed.md, one line, saying what is refused and that _-prefixed metadata keys still resolve.

Non-blocking

  • The message could say "names a private field of the entry" for this branch, so it reads differently from the method/property refusal.
  • One test through KBService.create_entry (or the CLI) would show the refusal reaches a caller as VALIDATION_FAILED.
  • #1633 adds _importance_given; with the structural test nothing here needs to wait for it.

For the record: this change is under pyrite/schema/ and changes what a write accepts, so after the next head it is read cold again and the maintainer merges it. No rebase is needed; the merge queue accepts a branch that is behind.

Recommendation: request changes.

@markramm markramm added the state: needs-author Changes requested on the current head: the next move is the author's label Nov 17, 2026
@Mohit-Gaur75

Copy link
Copy Markdown
Contributor Author

Thank you for the thorough and clear review! All three blocking points (and the non-blocking suggestions) have been addressed in commit 83998eea:

  1. Refuse only private dataclass fields (Option A): In FieldSchema.resolve_filename, separated dataclass fields into data_fields and private_fields. Only fields in private_fields raise ValidationError("which is a private field of entry..."). _-prefixed metadata keys (like _draft or _owner on kb.yaml-declared types) resolve from metadata, and absent _ names fall back to the default filename (None).
  2. Structural test: Parametrized over Entry and every class reachable via recursing Entry.__subclasses__() crossed with their private dataclass fields (143 pairs verified). Added controls for public fields, _draft metadata key on GenericEntry, absent _ names falling back to None, {date}--{slug}.md unchanged, and a test through KBService.create_entry verifying VALIDATION_FAILED.
  3. Changelog: Added changelog.d/1639-file-pattern-private-fields.fixed.md.

@markramm markramm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 4b92cff (conductor, 2026-11-17)

Automated: an Opus reviewer read 83998ee cold and then against the three items of the last review; the conductor checked its findings in the diff. Read-only: nothing was run. The test legs on this head had not finished when this was written, so this review does not claim the gate. The head moved to 4b92cff while it was being read: that is 83998ee plus a merge of dev, and its three changed files are byte-for-byte the same (git diff --quiet), so the review stands for this head.

Thank you for the fast turn. All three blocking items are met:

  1. Option A: met. TypeSchema.resolve_filename refuses field_name in private_fields, built from dataclasses.fields(entry) on the instance, so a subclass's own private fields (_entry_type on GenericEntry) are covered with no list of names in production code. A _ metadata key that is not a field still resolves, and an absent _ name still returns None. Format-spec spellings ({_x:>3}, {_x!r}) reach the refusal.
  2. Structural test: met. Pairs are computed at collection from the class tree, with the non-empty guard; by reading, every cell resolves or returns None on dev, so each fails without the change. The four controls are real controls. The old loop test that was removed asserted the over-broad rule; nothing is weakened.
  3. Fragment: met, and each claim in it matches the code. Both non-blocking suggestions (the message, the KBService.create_entry test) were taken too.

One change asked for, in tests/test_file_pattern.py, _collect_entry_private_field_pairs:

  • Entry.__subclasses__() also returns Entry subclasses that other test files define at module level (today: the probe classes in tests/test_declared_type_mapping.py and the classes in tests/test_field_aliases.py). So this test's parameter list depends on which test files were collected before it: it differs between a full run and running this file alone, and a test-only class that _build_test_entry cannot build would fail here, in a file its author never touched. Please keep only production classes, for example if not cls.__module__.startswith(("pyrite.", "pyrite_")): continue.
  • Sort by (c.__module__, c.__qualname__) instead of c.__name__. With names alone, two classes of the same name sort in set order, which differs between pytest-xdist workers and stops the run with "different tests were collected".

Optional, same push: data_fields still filters out _ names, but the refusal runs first, so data_fields = dataclass_fields - private_fields says the same with one filter.

For the record: this change is under pyrite/schema/ and changes what a write accepts, so the maintainer merges it. A merge of dev is not needed while the PR is mergeable; the merge queue accepts a branch that is behind.

Recommendation: merge after this one change.

@Mohit-Gaur75

Copy link
Copy Markdown
Contributor Author

Thank you for the review! Addressed in commit 7373d0c7:

  1. Production class filtering & deterministic sorting: In _collect_entry_private_field_pairs (tests/test_file_pattern.py), filtered classes to production models only (if not cls.__module__.startswith(("pyrite.", "pyrite_")): continue), and sorted deterministically by (c.__module__, c.__qualname__). Test IDs now use cls.__qualname__.
  2. Simplified data_fields: Updated FieldSchema.resolve_filename (pyrite/schema/field_schema.py) to data_fields = dataclass_fields - private_fields as suggested.

@markramm markramm left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 7373d0c (conductor, 2026-11-17)

Automated: read by the conductor from the pushed head (the change since 4b92cff: one commit, two files). Nothing was run. The checks on this head had not started when this was written, so this review does not claim the gate.

The one change asked for is made as asked: _collect_entry_private_field_pairs keeps only classes whose module starts with pyrite. or pyrite_, sorts by (module, qualname), and the ids use the qualname. Entry itself (pyrite.models.base) is still in the list. The optional tidy is in too: data_fields = dataclass_fields - private_fields, one filter, same sets.

Nothing else changed since the last review, so its three "met" items stand. Nothing further is needed from you. Thank you for three fast, exact rounds.

For the maintainer: this is under pyrite/schema/ and changes what a write accepts (a file_pattern naming a private entry field is refused with VALIDATION_FAILED), so the merge is yours once the gate is green on this head. The Opus read of the previous head found no blocking item; fragment present; #1639 is in 0.26.5.

Recommendation: merge.

@markramm markramm added state: needs-maintainer Reviewed clean; the maintainer merges (storage, server, schema or a public shape) and removed state: needs-author Changes requested on the current head: the next move is the author's labels Nov 17, 2026
@markramm
markramm added this pull request to the merge queue Nov 17, 2026
@markramm markramm added state: queued In the merge queue and removed state: needs-maintainer Reviewed clean; the maintainer merges (storage, server, schema or a public shape) labels Nov 17, 2026
Merged via the queue into pyrite-wiki:dev with commit 65d89b1 Nov 17, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state: queued In the merge queue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

kb.yaml file_pattern resolves private Entry fields ({_create_fields}, {_importance_given}) into file names

2 participants